feat(plugin-axe): add authentication support via setup script #1198

hanna-skryl · 2025-12-29T17:45:25Z

Add setupScript option for testing login-protected pages
Refactor browser management from standalone functions to AxeRunner class for cleaner lifecycle handling
Update Zod to v4.2.1 across all packages. This was required because Zod v4.1.5 fixed the z.function() type inference, which is now used for validating the setup function signature.
Refactor docsUrlSchema for Zod v4 compatibility
Test with a real Angular application behind username/password authentication

nx-cloud · 2025-12-29T17:46:47Z

View your CI Pipeline Execution ↗ for commit 7639b33

Command	Status	Duration	Result
`nx run ci:code-pushup -- merge-diffs --files=/h...`	✅ Succeeded	5s	View ↗
`nx run-many --targets=code-pushup --parallel=fa...`	✅ Succeeded	1m 26s	View ↗
`nx run-many --targets=code-pushup --parallel=fa...`	✅ Succeeded	12m 17s	View ↗
`nx run-many -t unit-test,int-test`	✅ Succeeded	3s	View ↗

☁️ Nx Cloud last updated this comment at 2026-01-14 19:15:23 UTC

pkg-pr-new · 2025-12-29T17:47:00Z

Open in StackBlitz

@code-pushup/ci

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/ci@1198

@code-pushup/cli

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/cli@1198

@code-pushup/core

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/core@1198

@code-pushup/create-cli

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/create-cli@1198

@code-pushup/models

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/models@1198

@code-pushup/nx-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/nx-plugin@1198

@code-pushup/axe-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/axe-plugin@1198

@code-pushup/coverage-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/coverage-plugin@1198

@code-pushup/eslint-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/eslint-plugin@1198

@code-pushup/js-packages-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/js-packages-plugin@1198

@code-pushup/jsdocs-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/jsdocs-plugin@1198

@code-pushup/lighthouse-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/lighthouse-plugin@1198

@code-pushup/typescript-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/typescript-plugin@1198

@code-pushup/utils

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/utils@1198

commit: 7639b33

github-actions · 2025-12-29T22:55:21Z

Code PushUp

🤨 Code PushUp report has both improvements and regressions – compared current commit 731580a with previous commit c66c042.

🕵️ See full comparison in Code PushUp portal 🔍

🏷️ Categories

🏷️ Category	⭐ Previous score	⭐ Current score	🔄 Score change
Documentation	🔴 35	🔴 39
Performance	🔴 36	🔴 34
Code coverage	🟢 93	🟢 93
Bug prevention	🟡 75	🟡 75
Axe Accessibility	🟡 88	🟡 88
Code style	🟢 99	🟢 99	–
Security	🟡 51	🟡 51	–
Updates	🟡 78	🟡 78	–
Type Safety	🟡 67	🟡 67	–
Miscellaneous	🟡 67	🟡 67	–
Accessibility	🟢 92	🟢 92	–
Best Practices	🟢 100	🟢 100	–
SEO	🟢 92	🟢 92	–

👍 1 group improved, 👎 2 groups regressed, 👍 6 audits improved, 👎 7 audits regressed, 13 audits changed without impacting score

🗃️ Groups

🔌 Plugin	🗃️ Group	⭐ Previous score	⭐ Current score
JSDocs coverage	Documentation coverage	🔴 35	🔴 39
Lighthouse	Performance	🔴 36	🔴 34
Code coverage	Code coverage metrics	🟢 93	🟢 93

31 other groups are unchanged.

🛡️ Audits

🔌 Plugin	🛡️ Audit	📏 Previous value	📏 Current value
Lighthouse	Initial server response time was short	🟥 Root document took 610 ms	🟩 Root document took 550 ms
JSDocs coverage	Methods coverage	🟥 13 undocumented methods	🟨 13 undocumented methods
JSDocs coverage	Classes coverage	🟥 7 undocumented classes	🟥 7 undocumented classes
Lighthouse	First Contentful Paint	🟥 3.1 s	🟥 3.3 s
Lighthouse	Speed Index	🟥 6.3 s	🟥 6.6 s
Lighthouse	Total Blocking Time	🟥 2,250 ms	🟥 2,850 ms
JSDocs coverage	Variables coverage	🟥 275 undocumented variables	🟥 268 undocumented variables
Lighthouse	Time to Interactive	🟥 13.3 s	🟥 13.9 s
JSDocs coverage	Functions coverage	🟥 591 undocumented functions	🟥 577 undocumented functions
Code coverage	Line coverage	🟩 92.8 %	🟩 92.6 %
JSDocs coverage	Types coverage	🟥 265 undocumented types	🟥 266 undocumented types
Code coverage	Branch coverage	🟨 89.1 %	🟨 89.1 %
Code coverage	Function coverage	🟩 94.3 %	🟩 94.3 %
Lighthouse	Avoids enormous network payloads	🟩 Total size was 2,107 KiB	🟩 Total size was 2,113 KiB
Lighthouse	Metrics	🟩 100%	🟩 100%
Lighthouse	JavaScript execution time	🟥 4.7 s	🟥 5.3 s
Lighthouse	Largest Contentful Paint	🟥 11.9 s	🟥 12.4 s
Lighthouse	Max Potential First Input Delay	🟥 1,070 ms	🟥 1,430 ms
Lighthouse	Uses efficient cache policy on static assets	🟨 31 resources found	🟨 31 resources found
Lighthouse	Server Backend Latencies	🟩 1,200 ms	🟩 1,450 ms
Lighthouse	Remove duplicate modules in JavaScript bundles	🟥 Potential savings of 99 KiB	🟥 Potential savings of 80 KiB
Lighthouse	Reduce unused CSS	🟥 Potential savings of 95 KiB	🟥 Potential savings of 95 KiB
Lighthouse	Network Round Trip Times	🟩 10 ms	🟩 70 ms
Lighthouse	Reduce unused JavaScript	🟥 Potential savings of 223 KiB	🟥 Potential savings of 223 KiB
JSDocs coverage	Properties coverage	🟥 23 undocumented properties	🟥 26 undocumented properties
Lighthouse	Minimizes main-thread work	🟥 11.3 s	🟥 11.3 s

653 other audits are unchanged.

github-actions · 2025-12-29T23:02:33Z

Code PushUp

🤨 Code PushUp report has both improvements and regressions – compared current commit 731580a with previous commit c66c042.

💼 Project `plugin-axe`

🤨 Code PushUp report has both improvements and regressions.

🕵️ See full comparison in Code PushUp portal 🔍

🏷️ Category	⭐ Previous score	⭐ Current score	🔄 Score change
Documentation	🟡 68	🟡 70
Code coverage	🟢 94	🟢 94

4 other categories are unchanged.

👍 1 group improved, 👎 1 group regressed, 👍 2 audits improved, 👎 3 audits regressed, 1 audit changed without impacting score

🗃️ Groups

🔌 Plugin	🗃️ Group	⭐ Previous score	⭐ Current score	🔄 Score change
JSDocs coverage	Documentation coverage	🟡 68	🟡 70
Code coverage	Code coverage metrics	🟢 94	🟢 94

13 other groups are unchanged.

🛡️ Audits

🔌 Plugin	🛡️ Audit	📏 Previous value	📏 Current value
JSDocs coverage	Properties coverage	🟩 0 undocumented properties	🟥 3 undocumented properties
JSDocs coverage	Functions coverage	🟥 24 undocumented functions	🟨 10 undocumented functions
JSDocs coverage	Variables coverage	🟥 19 undocumented variables	🟥 12 undocumented variables
Code coverage	Line coverage	🟨 72.2 %	🟨 70.5 %
Code coverage	Branch coverage	🟩 90.1 %	🟨 89.2 %
JSDocs coverage	Types coverage	🟥 8 undocumented types	🟥 9 undocumented types

437 other audits are unchanged.

💼 Project `create-cli`

😟 Code PushUp report has regressed.

🕵️ See full comparison in Code PushUp portal 🔍

🏷️ Category	⭐ Previous score	⭐ Current score	🔄 Score change
Code coverage	🟢 96	🟢 96

5 other categories are unchanged.

👎 1 group regressed, 👎 1 audit regressed

🗃️ Groups

🔌 Plugin	🗃️ Group	⭐ Previous score	⭐ Current score	🔄 Score change
Code coverage	Code coverage metrics	🟢 96	🟢 96

14 other groups are unchanged.

🛡️ Audits

🔌 Plugin	🛡️ Audit	📏 Previous value	📏 Current value	🔄 Value change
Code coverage	Branch coverage	🟨 88.2 %	🟨 87.5 %

443 other audits are unchanged.

💼 Project `models`

😟 Code PushUp report has regressed.

🕵️ See full comparison in Code PushUp portal 🔍

🏷️ Category	⭐ Previous score	⭐ Current score	🔄 Score change
Code coverage	🟢 94	🟢 94

3 other categories are unchanged.

👎 1 group regressed, 👎 1 audit regressed, 1 audit changed without impacting score

🗃️ Groups

🔌 Plugin	🗃️ Group	⭐ Previous score	⭐ Current score	🔄 Score change
Code coverage	Code coverage metrics	🟢 94	🟢 94

11 other groups are unchanged.

🛡️ Audits

🔌 Plugin	🛡️ Audit	📏 Previous value	📏 Current value	🔄 Value change
Code coverage	Branch coverage	🟩 94.2 %	🟩 94.1 %
Code coverage	Line coverage	🟩 99.4 %	🟩 99.4 %

435 other audits are unchanged.

11 other projects are unchanged.

BioPhoton

Added suggestion to reduce lines in tests.

packages/models/src/lib/implementation/schemas.ts

packages/plugin-axe/src/lib/runner/setup.int.test.ts

matejchalk

This is amazing work! 😍 The implementation is nice and clean, and the documentation should be fairly easy to use. I'm looking forward to trying it out! 🚀

matejchalk · 2026-01-13T17:17:21Z

packages/plugin-axe/src/lib/runner/setup.ts

+  const module: unknown = await import(absolutePath);
+  const validModule = await validateAsync(setupScriptModuleSchema, module, {
+    filePath: absolutePath,
+  });


I think it would be useful to log the setup script import. Since it's user-specified, there's a good chance of something going wrong (e.g., provides .ts file, invalid export). Grouping the dynamic import and validation in a logger task would help to provide more context.

Thanks for the pointer! Wrapped it in logger.task().

matejchalk · 2026-01-13T17:19:59Z

packages/plugin-axe/src/lib/runner/setup.int.test.ts

+      path.join(fixturesDir, 'valid-setup.ts'),
+    );
+
+    expect(typeof setupFn).toBe('function');


Vitest has a dedicated toBeTypeOf matcher that wraps typeof.

Suggested change

expect(typeof setupFn).toBe('function');

expect(setupFn).toBeTypeOf('function');

Used toBeFunction() from jest-extended instead - it seems more semantic.

matejchalk · 2026-01-13T17:24:53Z

e2e/plugin-axe-e2e/mocks/fixtures/code-pushup.config.ts

I'm anticipating that the setupScript option will be used very often, so it may be worth investing time into adding an E2E test scenario for it. 🤔 It doesn't have to be part of this PR, though.

Good idea, will add an E2E test for the setupScript option in a separate PR.

hanna-skryl self-assigned this Dec 29, 2025

github-actions bot added 📖 Project documentation improvements or additions to the project documentation 🔬 testing writing tests 🧩 models 🛠️ tooling 🧩 axe-plugin labels Dec 29, 2025

hanna-skryl added 2 commits December 29, 2025 17:28

chore: update zod to v4.2.1 across all packages

7de25ca

feat(plugin-axe): add authentication support via setup script

0d9fffb

hanna-skryl force-pushed the axe-auth-support branch from c44e7a7 to 0d9fffb Compare December 29, 2025 22:46

github-actions bot added the 🧩 eslint-plugin label Dec 29, 2025

docs(plugin-axe): improve documentation coverage

f0793e7

hanna-skryl marked this pull request as ready for review December 30, 2025 00:04

hanna-skryl requested review from BioPhoton and matejchalk as code owners December 30, 2025 00:04

BioPhoton approved these changes Jan 5, 2026

View reviewed changes

packages/models/src/lib/implementation/schemas.ts Show resolved Hide resolved

packages/plugin-axe/src/lib/runner/setup.int.test.ts Outdated Show resolved Hide resolved

refactor(plugin-axe): inline script paths in tests

85cf463

BioPhoton approved these changes Jan 5, 2026

View reviewed changes

github-actions bot added 🧩 cli 🧩 core 🧩 utils 🧩 coverage-plugin 🧩 typescript-plugin TypeScript plugin labels Jan 7, 2026

hanna-skryl force-pushed the axe-auth-support branch from c0cae0c to 85cf463 Compare January 7, 2026 14:49

github-actions bot removed 🧩 cli 🧩 core 🧩 utils labels Jan 7, 2026

github-actions bot removed 🧩 coverage-plugin 🧩 typescript-plugin TypeScript plugin labels Jan 7, 2026

Merge branch 'main' into axe-auth-support

2cadf0e

matejchalk approved these changes Jan 13, 2026

View reviewed changes

hanna-skryl and others added 3 commits January 14, 2026 13:50

fix(plugin-axe): update expectation for setup function

8dbbcac

refactor(plugin-axe): improve setup script logging

4380dfd

Merge branch 'main' into axe-auth-support

7639b33

	expect(typeof setupFn).toBe('function');
	expect(setupFn).toBeTypeOf('function');

feat(plugin-axe): add authentication support via setup script #1198

Are you sure you want to change the base?

feat(plugin-axe): add authentication support via setup script #1198

Uh oh!

Conversation

hanna-skryl commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nx-cloud bot commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pkg-pr-new bot commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code PushUp

🏷️ Categories

🗃️ Groups

🛡️ Audits

Uh oh!

github-actions bot commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code PushUp

💼 Project plugin-axe

🗃️ Groups

🛡️ Audits

💼 Project create-cli

🗃️ Groups

🛡️ Audits

💼 Project models

🗃️ Groups

🛡️ Audits

Uh oh!

BioPhoton left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

matejchalk left a comment

Choose a reason for hiding this comment

Uh oh!

matejchalk Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

hanna-skryl Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

matejchalk Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

hanna-skryl Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

matejchalk Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

hanna-skryl Jan 14, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hanna-skryl commented Dec 29, 2025 •

edited

Loading

nx-cloud bot commented Dec 29, 2025 •

edited

Loading

pkg-pr-new bot commented Dec 29, 2025 •

edited

Loading

github-actions bot commented Dec 29, 2025 •

edited

Loading

github-actions bot commented Dec 29, 2025 •

edited

Loading

💼 Project `plugin-axe`

💼 Project `create-cli`

💼 Project `models`